{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Collecting temperature data from an API\n",
    "\n",
    "## About the data\n",
    "In this notebook, we will be collecting daily temperature data from the [National Centers for Environmental Information (NCEI) API](https://www.ncdc.noaa.gov/cdo-web/webservices/v2). We will use the Global Historical Climatology Network - Daily (GHCND) data set; see the documentation [here](https://www1.ncdc.noaa.gov/pub/data/cdo/documentation/GHCND_documentation.pdf).\n",
    "\n",
    "*Note: The NCEI is part of the National Oceanic and Atmospheric Administration (NOAA) and, as you can see from the URL for the API, this resource was created when the NCEI was called the NCDC. Should the URL for this resource change in the future, you can search for the NCEI weather API to find the updated one.*\n",
    "\n",
    "## Using the NCEI API\n",
    "Paste your token below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "def make_request(endpoint, payload=None):\n",
    "    \"\"\"\n",
    "    Make a request to a specific endpoint on the weather API\n",
    "    passing headers and optional payload.\n",
    "    \n",
    "    Parameters:\n",
    "        - endpoint: The endpoint of the API you want to \n",
    "                    make a GET request to.\n",
    "        - payload: A dictionary of data to pass along \n",
    "                   with the request.\n",
    "    \n",
    "    Returns:\n",
    "        Response object.\n",
    "    \"\"\"\n",
    "    return requests.get(\n",
    "        f'https://www.ncdc.noaa.gov/cdo-web/api/v2/{endpoint}',\n",
    "        headers={\n",
    "            'token': 'PASTE_YOUR_TOKEN_HERE'\n",
    "        },\n",
    "        params=payload\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## See what datasets are available\n",
    "We can make requests to the `datasets` endpoint to see what datasets are available. We also pass in a dictionary for the payload to get datsets that have data after the start date of October 1, 2018."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "200"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# see what datasets are available\n",
    "response = make_request('datasets', {'startdate':'2018-10-01'})\n",
    "response.status_code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Status code of `200` means everything is OK. More codes can be found [here](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes).\n",
    "\n",
    "### Get the keys of the result\n",
    "The result is a JSON object which we can access with the `json()` method of our `Response` object. JSON objects can be treated like dictionaries, so we can access the `keys()` just like we would a dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['metadata', 'results'])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.json().keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `metadata` of the JSON response will tell us information about the request and data we got back:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'resultset': {'offset': 1, 'count': 11, 'limit': 25}}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.json()['metadata']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Figure out what data is in the result\n",
    "The `results` key contains the data we requested. This is a list of what would be rows in our dataframe. Each entry in the list is a dictionary, so we can look at the keys to get the fields:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['uid', 'mindate', 'maxdate', 'name', 'datacoverage', 'id'])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.json()['results'][0].keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parse the result\n",
    "We don't want all those fields, so we will use a list comphrension to take only the `id` and `name` fields out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('GHCND', 'Daily Summaries'),\n",
       " ('GSOM', 'Global Summary of the Month'),\n",
       " ('GSOY', 'Global Summary of the Year'),\n",
       " ('NEXRAD2', 'Weather Radar (Level II)'),\n",
       " ('NEXRAD3', 'Weather Radar (Level III)'),\n",
       " ('NORMAL_ANN', 'Normals Annual/Seasonal'),\n",
       " ('NORMAL_DLY', 'Normals Daily'),\n",
       " ('NORMAL_HLY', 'Normals Hourly'),\n",
       " ('NORMAL_MLY', 'Normals Monthly'),\n",
       " ('PRECIP_15', 'Precipitation 15 Minute'),\n",
       " ('PRECIP_HLY', 'Precipitation Hourly')]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[(data['id'], data['name']) for data in response.json()['results']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Figure out which data category we want\n",
    "The `GHCND` data containing daily summaries is what we want. Now we need to make another request to figure out which data categories we want to collect. This is the `datacategories` endpoint. We have to pass the `datasetid` for `GHCND` as the payload so the API knows which dataset we are asking about:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "200"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get data category id\n",
    "response = make_request(\n",
    "    'datacategories',\n",
    "    payload={\n",
    "        'datasetid' : 'GHCND'\n",
    "    }\n",
    ")\n",
    "response.status_code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we know the API gives us a `metadata` and a `results` key in each response, we can see what is in the `results` portion of the JSON response:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'name': 'Evaporation', 'id': 'EVAP'},\n",
       " {'name': 'Land', 'id': 'LAND'},\n",
       " {'name': 'Precipitation', 'id': 'PRCP'},\n",
       " {'name': 'Sky cover & clouds', 'id': 'SKY'},\n",
       " {'name': 'Sunshine', 'id': 'SUN'},\n",
       " {'name': 'Air Temperature', 'id': 'TEMP'},\n",
       " {'name': 'Water', 'id': 'WATER'},\n",
       " {'name': 'Wind', 'id': 'WIND'},\n",
       " {'name': 'Weather Type', 'id': 'WXTYPE'}]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.json()['results']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Grab the data type ID for the Temperature category\n",
    "We will be working with temperatures, so we want the `TEMP` data category. Now, we need to find the `datatypes` to collect. For this, we use the `datatypes` endpoint and provide the `datacategoryid` which was `TEMP`. We also specify a limit for the number of `datatypes` to return with the payload. If there are more than this we can make another request later, but for now, we just want to pick a few out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "200"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get data type id\n",
    "response = make_request(\n",
    "    'datatypes',\n",
    "    payload={\n",
    "        'datacategoryid' : 'TEMP', \n",
    "        'limit' : 100\n",
    "    }\n",
    ")\n",
    "response.status_code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can grab the `id` and `name` fields for each of the entries in the `results` portion of the data. The fields we are interested in are at the bottom:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('MNTM', 'Monthly mean temperature'),\n",
       " ('TAVG', 'Average Temperature.'),\n",
       " ('TMAX', 'Maximum temperature'),\n",
       " ('TMIN', 'Minimum temperature'),\n",
       " ('TOBS', 'Temperature at the time of observation')]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[(datatype['id'], datatype['name']) for datatype in response.json()['results']][-5:] # look at the last 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Determine which Location Category we want\n",
    "Now that we know which `datatypes` we will be collecting, we need to find the location to use. First, we need to figure out the location category. This is obtained from the `locationcategories` endpoint by passing the `datasetid`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "200"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get location category id \n",
    "response = make_request(\n",
    "    'locationcategories', \n",
    "    {\n",
    "        'datasetid' : 'GHCND'\n",
    "    }\n",
    ")\n",
    "response.status_code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use `pprint` to print dictionaries in an easier-to-read format. After doing so, we can see there are 12 different location categories, but we are only interested in `CITY`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'metadata': {'resultset': {'count': 12, 'limit': 25, 'offset': 1}},\n",
      " 'results': [{'id': 'CITY', 'name': 'City'},\n",
      "             {'id': 'CLIM_DIV', 'name': 'Climate Division'},\n",
      "             {'id': 'CLIM_REG', 'name': 'Climate Region'},\n",
      "             {'id': 'CNTRY', 'name': 'Country'},\n",
      "             {'id': 'CNTY', 'name': 'County'},\n",
      "             {'id': 'HYD_ACC', 'name': 'Hydrologic Accounting Unit'},\n",
      "             {'id': 'HYD_CAT', 'name': 'Hydrologic Cataloging Unit'},\n",
      "             {'id': 'HYD_REG', 'name': 'Hydrologic Region'},\n",
      "             {'id': 'HYD_SUB', 'name': 'Hydrologic Subregion'},\n",
      "             {'id': 'ST', 'name': 'State'},\n",
      "             {'id': 'US_TERR', 'name': 'US Territory'},\n",
      "             {'id': 'ZIP', 'name': 'Zip Code'}]}\n"
     ]
    }
   ],
   "source": [
    "import pprint\n",
    "pprint.pprint(response.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get NYC Location ID\n",
    "In order to find the location ID for New York, we need to search through all the cities available. Since we can ask the API to return the cities sorted, we can use binary search to find New York quickly without having to make many requests or request lots of data at once. The following function makes the first request to see how big the list of cities is and looks at the first value. From there it decides if it needs to move towards the beginning or end of the list by comparing the city we are looking for to others alphabetically. Each time it makes a request it can rule out half of the remaining data to search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_item(name, what, endpoint, start=1, end=None):\n",
    "    \"\"\"\n",
    "    Grab the JSON payload for a given field by name using binary search.\n",
    "\n",
    "    Parameters:\n",
    "        - name: The item to look for.\n",
    "        - what: Dictionary specifying what the item in `name` is.\n",
    "        - endpoint: Where to look for the item.\n",
    "        - start: The position to start at. We don't need to touch this, but the\n",
    "                 function will manipulate this with recursion.\n",
    "        - end: The last position of the cities. Used to find the midpoint, but\n",
    "               like `start` this is not something we need to worry about.\n",
    "\n",
    "    Returns:\n",
    "        Dictionary of the information for the item if found otherwise \n",
    "        an empty dictionary.\n",
    "    \"\"\"\n",
    "    # find the midpoint which we use to cut the data in half each time\n",
    "    mid = (start + (end if end else 1)) // 2\n",
    "    \n",
    "    # lowercase the name so this is not case-sensitive\n",
    "    name = name.lower()\n",
    "    \n",
    "    # define the payload we will send with each request\n",
    "    payload = {\n",
    "        'datasetid' : 'GHCND',\n",
    "        'sortfield' : 'name',\n",
    "        'offset' : mid, # we will change the offset each time\n",
    "        'limit' : 1 # we only want one value back\n",
    "    }\n",
    "    \n",
    "    # make our request adding any additional filter parameters from `what`\n",
    "    response = make_request(endpoint, {**payload, **what})\n",
    "    \n",
    "    if response.ok:\n",
    "        # if response is ok, grab the end index from the response metadata the first time through\n",
    "        end = end if end else response.json()['metadata']['resultset']['count']\n",
    "        \n",
    "        # grab the lowercase version of the current name\n",
    "        current_name = response.json()['results'][0]['name'].lower()\n",
    "        \n",
    "        # if what we are searching for is in the current name, we have found our item\n",
    "        if name in current_name:\n",
    "            return response.json()['results'][0] # return the found item\n",
    "        else:\n",
    "            if start >= end: \n",
    "                # if our start index is greater than or equal to our end, we couldn't find it\n",
    "                return {}\n",
    "            elif name < current_name:\n",
    "                # our name comes before the current name in the alphabet, so we search further to the left\n",
    "                return get_item(name, what, endpoint, start, mid - 1)\n",
    "            elif name > current_name:\n",
    "                # our name comes after the current name in the alphabet, so we search further to the right\n",
    "                return get_item(name, what, endpoint, mid + 1, end)    \n",
    "    else:\n",
    "        # response wasn't ok, use code to determine why\n",
    "        print(f'Response not OK, status: {response.status_code}')\n",
    "\n",
    "def get_location(name):\n",
    "    \"\"\"\n",
    "    Grab the JSON payload for the location by name using binary search.\n",
    "\n",
    "    Parameters:\n",
    "        - name: The city to look for.\n",
    "\n",
    "    Returns:\n",
    "        Dictionary of the information for the city if found otherwise \n",
    "        an empty dictionary.\n",
    "    \"\"\"\n",
    "    return get_item(name, {'locationcategoryid' : 'CITY'}, 'locations')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we use binary search to find New York, we find it in just 8 requests despite it being close to the middle of 1,983 entries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'mindate': '1869-01-01',\n",
       " 'maxdate': '2019-05-02',\n",
       " 'name': 'New York, NY US',\n",
       " 'datacoverage': 1,\n",
       " 'id': 'CITY:US360019'}"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get NYC id \n",
    "nyc = get_location('New York')\n",
    "nyc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get the station ID for Central Park\n",
    "The most granular data is found at the station level:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'elevation': 42.7,\n",
       " 'mindate': '1869-01-01',\n",
       " 'maxdate': '2019-05-01',\n",
       " 'latitude': 40.77898,\n",
       " 'name': 'NY CITY CENTRAL PARK, NY US',\n",
       " 'datacoverage': 1,\n",
       " 'id': 'GHCND:USW00094728',\n",
       " 'elevationUnit': 'METERS',\n",
       " 'longitude': -73.96925}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "central_park = get_item('NY City Central Park', {'locationid' : nyc['id']}, 'stations')\n",
    "central_park"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Request the temperature data\n",
    "Finally, we have everything we need to make our request for the New York temperature data. For this we use the `data` endpoint and provide all the parameters we picked up throughout our exploration of the API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "200"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get NYC daily summaries data \n",
    "response = make_request(\n",
    "    'data', \n",
    "    {\n",
    "        'datasetid' : 'GHCND',\n",
    "        'stationid' : central_park['id'],\n",
    "        'locationid' : nyc['id'],\n",
    "        'startdate' : '2018-10-01',\n",
    "        'enddate' : '2018-10-31',\n",
    "        'datatypeid' : ['TMIN', 'TMAX', 'TOBS'], # temperature at time of observation, min, and max\n",
    "        'units' : 'metric',\n",
    "        'limit' : 1000\n",
    "    }\n",
    ")\n",
    "response.status_code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a DataFrame\n",
    "The Central Park station only has the daily minimum and maximum temperatures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>attributes</th>\n",
       "      <th>datatype</th>\n",
       "      <th>date</th>\n",
       "      <th>station</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>,,W,2400</td>\n",
       "      <td>TMAX</td>\n",
       "      <td>2018-10-01T00:00:00</td>\n",
       "      <td>GHCND:USW00094728</td>\n",
       "      <td>24.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>,,W,2400</td>\n",
       "      <td>TMIN</td>\n",
       "      <td>2018-10-01T00:00:00</td>\n",
       "      <td>GHCND:USW00094728</td>\n",
       "      <td>17.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>,,W,2400</td>\n",
       "      <td>TMAX</td>\n",
       "      <td>2018-10-02T00:00:00</td>\n",
       "      <td>GHCND:USW00094728</td>\n",
       "      <td>25.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>,,W,2400</td>\n",
       "      <td>TMIN</td>\n",
       "      <td>2018-10-02T00:00:00</td>\n",
       "      <td>GHCND:USW00094728</td>\n",
       "      <td>18.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>,,W,2400</td>\n",
       "      <td>TMAX</td>\n",
       "      <td>2018-10-03T00:00:00</td>\n",
       "      <td>GHCND:USW00094728</td>\n",
       "      <td>23.3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  attributes datatype                 date            station  value\n",
       "0   ,,W,2400     TMAX  2018-10-01T00:00:00  GHCND:USW00094728   24.4\n",
       "1   ,,W,2400     TMIN  2018-10-01T00:00:00  GHCND:USW00094728   17.2\n",
       "2   ,,W,2400     TMAX  2018-10-02T00:00:00  GHCND:USW00094728   25.0\n",
       "3   ,,W,2400     TMIN  2018-10-02T00:00:00  GHCND:USW00094728   18.3\n",
       "4   ,,W,2400     TMAX  2018-10-03T00:00:00  GHCND:USW00094728   23.3"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.DataFrame(response.json()['results'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We didn't get TOBS because the station doesn't measure that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['TMAX', 'TMIN'], dtype=object)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.datatype.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Despite showing up in the data as measuring it... Real-world data is dirty!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found!\n"
     ]
    }
   ],
   "source": [
    "if get_item(\n",
    "    'NY City Central Park', {'locationid' : nyc['id'], 'datatypeid': 'TOBS'}, 'stations'\n",
    "):\n",
    "    print('Found!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using a different station\n",
    "Let's use LaGuardia airport instead. It contains `TAVG` (average daily temperature):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'elevation': 3.4,\n",
       " 'mindate': '1939-10-07',\n",
       " 'maxdate': '2019-05-02',\n",
       " 'latitude': 40.7792,\n",
       " 'name': 'LAGUARDIA AIRPORT, NY US',\n",
       " 'datacoverage': 1,\n",
       " 'id': 'GHCND:USW00014732',\n",
       " 'elevationUnit': 'METERS',\n",
       " 'longitude': -73.88}"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "laguardia = get_item(\n",
    "    'LaGuardia', {'locationid' : nyc['id']}, 'stations'\n",
    ")\n",
    "laguardia"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We make our request using the LaGuardia airport station this time and ask for `TAVG` instead of `TOBS`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "200"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get NYC daily summaries data \n",
    "response = make_request(\n",
    "    'data', \n",
    "    {\n",
    "        'datasetid' : 'GHCND',\n",
    "        'stationid' : laguardia['id'],\n",
    "        'locationid' : nyc['id'],\n",
    "        'startdate' : '2018-10-01',\n",
    "        'enddate' : '2018-10-31',\n",
    "        'datatypeid' : ['TMIN', 'TMAX', 'TAVG'], # temperature at time of observation, min, and max\n",
    "        'units' : 'metric',\n",
    "        'limit' : 1000\n",
    "    }\n",
    ")\n",
    "response.status_code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The request was successful, so let's make a dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>attributes</th>\n",
       "      <th>datatype</th>\n",
       "      <th>date</th>\n",
       "      <th>station</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>H,,S,</td>\n",
       "      <td>TAVG</td>\n",
       "      <td>2018-10-01T00:00:00</td>\n",
       "      <td>GHCND:USW00014732</td>\n",
       "      <td>21.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>,,W,2400</td>\n",
       "      <td>TMAX</td>\n",
       "      <td>2018-10-01T00:00:00</td>\n",
       "      <td>GHCND:USW00014732</td>\n",
       "      <td>25.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>,,W,2400</td>\n",
       "      <td>TMIN</td>\n",
       "      <td>2018-10-01T00:00:00</td>\n",
       "      <td>GHCND:USW00014732</td>\n",
       "      <td>18.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>H,,S,</td>\n",
       "      <td>TAVG</td>\n",
       "      <td>2018-10-02T00:00:00</td>\n",
       "      <td>GHCND:USW00014732</td>\n",
       "      <td>22.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>,,W,2400</td>\n",
       "      <td>TMAX</td>\n",
       "      <td>2018-10-02T00:00:00</td>\n",
       "      <td>GHCND:USW00014732</td>\n",
       "      <td>26.1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  attributes datatype                 date            station  value\n",
       "0      H,,S,     TAVG  2018-10-01T00:00:00  GHCND:USW00014732   21.2\n",
       "1   ,,W,2400     TMAX  2018-10-01T00:00:00  GHCND:USW00014732   25.6\n",
       "2   ,,W,2400     TMIN  2018-10-01T00:00:00  GHCND:USW00014732   18.3\n",
       "3      H,,S,     TAVG  2018-10-02T00:00:00  GHCND:USW00014732   22.7\n",
       "4   ,,W,2400     TMAX  2018-10-02T00:00:00  GHCND:USW00014732   26.1"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(response.json()['results'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We should check we got what we wanted: 31 entries for TAVG, TMAX, and TMIN (1 per day):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TMAX    31\n",
       "TAVG    31\n",
       "TMIN    31\n",
       "Name: datatype, dtype: int64"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.datatype.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Write the data to a CSV file for use in other notebooks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv('data/nyc_temperatures.csv', index=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
